{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zwBCE43Cv3PH"
   },
   "source": [
    "## Tensorflow怎样保存与加载模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YQB7yiF6v9GR"
   },
   "source": [
    "#### 背景：模型训练和预估服务\n",
    "* 训练模型一般不是目标，在线预估服务才是\n",
    "* 一般会有一个程序训练模型，训练好之后把模型保存，在线预估加载模型实现服务\n",
    "* 模型一般包括模型结构、训练好的模型参数两部分组成\n",
    "\n",
    "#### 保存模型的方法\n",
    "* 模型结构+参数打包保存和加载\n",
    "* 模型结构存储到json文件，模型参数保存到h5文件\n",
    "* 模型结构存储到yaml文件，模型参数保存到h5文件\n",
    "\n",
    "其实，模型参数甚至也会存储到redis、mysql等，各个web服务可以随意更新加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 1. 准备和训练一个模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5IoRbCA2n0_V"
   },
   "source": [
    "#### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UEfJ8TcMpe-2"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"./datas/heart/heart.csv\")\n",
    "\n",
    "# 把thal列变成数字编码\n",
    "df['thal'] = pd.Categorical(df['thal'])\n",
    "df['thal'] = df['thal'].cat.codes\n",
    "\n",
    "# 要预测的目标，这是个二分类问题\n",
    "target = df.pop('target')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LmCl5R5C2IKo"
   },
   "source": [
    "#### 构建dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W6Yc-D3aqyBb"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:57.600516: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA\n",
      "To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-03-25 16:09:57.600911: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 12. Tune using inter_op_parallelism_threads for best performance.\n"
     ]
    }
   ],
   "source": [
    "# 构建dataset，其实是把pandas数据转换成numpy数组进行转换的\n",
    "dataset = tf.data.Dataset.from_tensor_slices((df.values, target.values))\n",
    "# Shuffle and batch the dataset.\n",
    "train_dataset = dataset.shuffle(len(df)).batch(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 64.    1.    4.  145.  212.    0.    2.  132.    0.    2.    2.    2.\n",
      "    2. ]\n",
      " [ 42.    1.    1.  148.  244.    0.    2.  178.    0.    0.8   1.    2.\n",
      "    3. ]\n",
      " [ 62.    0.    4.  160.  164.    0.    2.  145.    0.    6.2   3.    3.\n",
      "    4. ]\n",
      " [ 59.    1.    4.  170.  326.    0.    2.  140.    1.    3.4   3.    0.\n",
      "    4. ]]\n"
     ]
    }
   ],
   "source": [
    "# 用于一会的测试\n",
    "for x, y in train_dataset.take(1):\n",
    "    input_data = x.numpy()\n",
    "    print(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "R3dQ-83Ztsgl"
   },
   "source": [
    "#### 搭建训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "76/76 [==============================] - 1s 14ms/step - loss: 18.2304 - accuracy: 0.6436\n",
      "Epoch 2/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 3.7232 - accuracy: 0.5512\n",
      "Epoch 3/10\n",
      " 1/76 [..............................] - ETA: 0s - loss: 0.6060 - accuracy: 0.7500"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:58.798956: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:09:58.936412: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 1ms/step - loss: 1.7737 - accuracy: 0.6205\n",
      "Epoch 4/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 1.0381 - accuracy: 0.6700\n",
      "Epoch 5/10\n",
      "39/76 [==============>...............] - ETA: 0s - loss: 0.6703 - accuracy: 0.6731"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:59.045061: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:09:59.156678: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 2ms/step - loss: 0.6750 - accuracy: 0.6799\n",
      "Epoch 6/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 0.6340 - accuracy: 0.7195\n",
      "Epoch 7/10\n",
      " 1/76 [..............................] - ETA: 0s - loss: 1.0259 - accuracy: 0.5000"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:59.277655: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:09:59.426002: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 2ms/step - loss: 0.5864 - accuracy: 0.7492\n",
      "Epoch 8/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 0.5650 - accuracy: 0.7360\n",
      "Epoch 9/10\n",
      " 1/76 [..............................] - ETA: 0s - loss: 1.6693 - accuracy: 0.5000"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:59.599700: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:09:59.762851: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 2ms/step - loss: 0.5438 - accuracy: 0.7459\n",
      "Epoch 10/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 0.5333 - accuracy: 0.7822\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:09:59.906969: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:10:00.031307: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fd59752d518>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    tf.keras.layers.Dense(10, input_shape=(df.shape[1],)),\n",
    "    tf.keras.layers.Dense(10, activation='relu'),\n",
    "    tf.keras.layers.Dense(1)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_dataset, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法1：把模型结构和模型参数一起保存"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save(\"./models/heart_model_method1.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'str' object has no attribute 'decode'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m/var/folders/29/dh100_vx1w7961c6rvp66qf00000gn/T/ipykernel_36414/3281078383.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmodel_total\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"./models/heart_model_method1.h5\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/opt/anaconda3/envs/pytensorflow/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py\u001b[0m in \u001b[0;36mload_model\u001b[0;34m(filepath, custom_objects, compile)\u001b[0m\n\u001b[1;32m    144\u001b[0m   if (h5py is not None and (\n\u001b[1;32m    145\u001b[0m       isinstance(filepath, h5py.File) or h5py.is_hdf5(filepath))):\n\u001b[0;32m--> 146\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mhdf5_format\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload_model_from_hdf5\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_objects\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcompile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    147\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    148\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstring_types\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/envs/pytensorflow/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py\u001b[0m in \u001b[0;36mload_model_from_hdf5\u001b[0;34m(filepath, custom_objects, compile)\u001b[0m\n\u001b[1;32m    164\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mmodel_config\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    165\u001b[0m       \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'No model found in config file.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 166\u001b[0;31m     \u001b[0mmodel_config\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mjson\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel_config\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'utf-8'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    167\u001b[0m     model = model_config_lib.model_from_config(model_config,\n\u001b[1;32m    168\u001b[0m                                                custom_objects=custom_objects)\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'str' object has no attribute 'decode'"
     ]
    }
   ],
   "source": [
    "model_total = tf.keras.models.load_model(\"./models/heart_model_method1.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 一个batch\n",
    "input_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_total.predict(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法2：模型结构保存到json，模型参数保存到h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型结构保存到json文件\n",
    "open(\"./models/heart_model_json.json\", \"w\").write(model.to_json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型参数保存到.h5文件\n",
    "model.save_weights(\"./models/heart_model_json.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载模型\n",
    "model_json = tf.keras.models.model_from_json(open(\"./models/heart_model_json.json\").read())\n",
    "model_json.load_weights(\"./models/heart_model_json.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 注意需要重新编译模型\n",
    "model_json.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 实现预估\n",
    "model_json.predict(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法3：模型结构保存到yaml，模型参数保存到h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型结构保存到yaml文件\n",
    "open(\"./models/heart_model_yaml.yaml\", \"w\").write(model.to_yaml())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型参数保存到.h5文件\n",
    "model.save_weights(\"./models/heart_model_yaml.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载模型\n",
    "model_yaml = tf.keras.models.model_from_yaml(open(\"./models/heart_model_yaml.yaml\").read())\n",
    "model_yaml.load_weights(\"./models/heart_model_yaml.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 注意需要重新编译模型\n",
    "model_yaml.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 实现预估\n",
    "model_yaml.predict(input_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.DataFrame(input_data).to_json(orient='values')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "pandas_dataframe.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
